Changing Sociocultural Dynamics and Implications for National Security: Proceedings of a Workshop by unknow

Author:unknow
Language: eng
Format: epub
Publisher: The National Academies Press
Published: 2018-05-15T00:00:00+00:00

DISCUSSION

Following the panel presentations, workshop participants discussed some of the ideas raised. The discussion, moderated by Mark Liberman, focused on several themes: (1) the need for methods that are adaptable, timely, and rigorous; (2) communicating with decision makers and building trust; (3) roles and limitations of automated and human data analysis; and (4) investments in data sources and data sharing.

Need for Adaptability, Speed, and Rigor

One participant from the Intelligence Community (IC) noted that in that community, methods need to be adaptable to new situations and data sources, produce timely results, and be rigorous enough to provide confidence in the findings. Two panelists discussed the trade-offs of speed and rigor and ways to address them. Resnik said, “So much of what we do as data-driven scientists involves developing automatic methods, but I am firmly convinced that when you want to do things in the real world, you need to spend more time focusing on the human in the loop.” He added that the combination of automation and human analysis balances bottom-up approaches, in which conclusions are reached based on patterns that emerge from the data collected, and top-down approaches, in which humans help define categories used to interpret these patterns. When the two approaches are used together, he asserted, the limitations of either alone are balanced, lending confidence to the results.

Broniatowski noted that each method comes with pros and cons that must be weighed against the primary considerations of the researchers. For example, he observed, social media data can be collected very quickly, but rigorous methods for their analysis comparable to survey research methods have not yet been established. Alternatively, he suggested, when rapid results are required, such as during an outbreak of infectious disease, using a rigorously designed survey of people’s understanding and actions might not produce results in time, even if it yields robust findings. For these reasons, he argued, both researchers and consumers of research should be aware of the trade-offs entailed in selecting one approach over another. He added that being able to communicate these trade-offs and the nature of uncertainty to decision makers is of critical importance. This means, he elaborated, that analysts need to be clear on the central message, the gist, that they want to convey about the data while accurately portraying the degree of confidence in that message in an understandable way.

Broniatowski and Liberman also addressed the need for methods adaptable to new questions and contexts. Broniatowski explained that flexibility can be built into the structures that govern the flow of information in organizations, suggesting that resources and research are available to guide the examination and design of such structures. Liberman stated that the issue of generalizability—whether findings from one context hold true in another—is linked to the issue of communicating clearly to instill confidence in decision makers.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.